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ABSTRACT 
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experiences; develop theoretically and empirically based constructs of 
teacher preparation, induction, knowledge, beliefs, and practice; and 
determine relationships among three sets of variables (teacher preparation, 
teaching knowledge and practice, and student learning) . The survey was 
administered in Tennessee and Connecticut to first, second, and third year 
teachers of grades 3-8 who taught reading and/or mathematics. This paper 
reports findings from Tennessee only, focusing on mathematics instruction. By 
linking the database of survey responses from Tennessee beginning teachers to 
measures of student academic growth, researchers were able to examine 
relationships between teacher preparation/practice variables and measures of 
teacher effectiveness based on student learning. Preliminary findings 
indicate that beginning teachers were more successful in teaching mathematics 
if they: believed that teachers can have an impact on student learning; 
helped students make sense out of mathematics; learned how to do this in 
their teacher preparation program; and received continued support in their 
first teaching years from an experienced mentor. Appended is the theoretical 
model. (Contains 36 references.) (SM) 
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This paper presents an initial and partial analysis of a Beginning Teacher Survey Study. It 
describes the study's overall rationale, design, theoretical framework, and research questions as 
well as a few, preliminary findings. A companion paper prepared for the symposium by Mark 
Reckase ("Methodological Issues in Describing the Features and Qualities of Teacher Education 
Programs") describes the methodology in greater depth. A final report of the project will be 
submitted to OERI and the Ford Foundation later the spring (May/June 2001). 

The study has been supported by the U.S. Department of Education, Office of 
Educational Research and Improvement, since January 1999. Noted scholars from across the 
country contributed to the overall research design and development of survey items. A grant from 
the Ford Foundation covered costs of administering the survey. The National Education 
Association, Tennessee Education Association, and three local affiliates were active supporters, 
contributing letters of endorsement, access to databases, and resources, at cost, from the NEA 
Professional Library. National Computer Systems was hired to design, print, and scan responses 
to the survey, which was administered to beginning teachers during the Spring Semester, 2000. 
William Sanders, formerly Professor and Director of The University of Tennessee Value-Added 
Research and Assessment Center and currently with SASinSchool™ assisted with the sampling 
design and data analysis for the Tennessee sample. Ray Pecheone, Katie Moirs and their 
colleagues at the Connecticut State Department of Education did the same for the Connecticut 
sample. 

Purpose 

The Beginning Teacher Survey Study, has three independent, but inter- related goals: 

• To develop a survey that would provide useful information from beginning teachers 
about their preservice and induction learning experiences and opportunities 

• To develop theoretically and empirically-based constructs of teacher preparation, 
induction, teacher knowledge and beliefs, and teaching practice 

• To determine relationships among three sets of variables: (a) teacher preparation, (b) 
teaching knowledge and practice, and (c) student learning. 

Until now, research on the quality and impact of teacher preparation has relied primarily 
on indicators of effectiveness such as expert judgment, observation of practice, interviews of 
graduates, and surveys of principals and supervisors (Kennedy, 1996; Zeichner, 1999). But with 
growing concern about educational quality and equality, policy makers are seeking more direct 
evidence of teaching quality-impact on student learning— despite the methodological problems 
posed (see for example Berk, 1988). This survey was designed to provide information about the 
quality and impact of both university -based, pre-service teacher preparation and the continued 
support and induction experiences in the first years of teaching. 

To provide information about the effectiveness of teacher and teacher education programs 
based on student learning, the survey was administered in two states, Tennessee and 
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Connecticut, from which we could obtain measures of student learning attributable to classroom 
teachers. These measures were then merged with the survey results. In this paper, we report 
only on findings from the Tennessee sample. Our colleagues from the Connecticut State 
Department of Education discuss the Connecticut results in their "Policy-Practice Perspective" 
paper. By linking the database of survey responses from Tennessee beginning teachers to 
measures of student academic growth, we were able to examine the relationship between teacher 
preparation/practice variables and measures of teacher effectiveness based on student learning. 

Since the ultimate goal of the study was to determine the impact professional preparation 
and support systems have on the effective practice of beginning teachers, the study addressed 
two main questions: 

(1) Does formal teacher preparation have an impact on teaching knowledge, beliefs, 
and practices in ways that positively affect student learning? 

(2) If so, what characteristics of that preparation are most important? 

Theoretical Model 

The theoretical model underlying the study posits a direct, linear relationship between (a) 
teacher learning opportunities, (b) teaching knowledge, beliefs, and practices, and (c) student 
learning. Teacher learning opportunities will impact student learning indirectly— through teaching 
practice itself. Since there are numerous other influences on teaching knowledge and practice 
(innate ability; general educational opportunities; family/community influence; school context; 
students' readiness to learn) and on student learning (family background, motivation, prior 
learning, educational resources), we anticipate that the impact of our predictor variables will be 
relatively small, no more than 5% of the explained variance. This effect size is, however, 
potentially significant. As previous studies have shown, high quality teachers have enormous 
impact on student learning (Ferguson, 1991; Greenwald, Hedges & Laine, 1996; Sanders & Horn; 
1998). 



One of our first task in designing the study was to conceptualize the components of 
teacher preparation, knowledge, and practice likely to have the most impact on student learning 
as measured by standardized tests of elementary-level reading and mathematics. Figure 1 depicts 
the theoretical model that informed the development of the survey and analysis of responses. As 
seen in that diagram, we assume that teacher preparation has an impact on teaching knowledge, 
beliefs, and practices, which then affect student learning. The overall quality and impact of 
teacher preparation is influenced by structural and conceptual features of a candidate's program. 
In addition to the preservice aspect of preparation, teacher effectiveness is also influenced by in- 
service opportunities to learn. During the beginning years of teaching, these professional 
development experiences are generally thought of, and often formally organized, as teacher 
induction. 



But formal teacher preparation is not the only influence on teaching effectiveness. In fact, 
there is reason to believe that it is not even the most significant influence. An individual does not 
come to teaching as a blank slate. Each already has well formed beliefs, dispositions and 
personalities. In teaching, more than any other profession, personal beliefs and knowledge 
schemas about future work are established long before any formal preparation begins (Dunkin & 
Biddle, 1974; Lortie, 1975). Therefore, the theoretical model considers the influence of individual 
differences that precede teacher preparation on measures of teacher effectiveness. In addition, 
school contexts impact the quality of teaching. This influence is shown in Figure 1 as Classroom 
and School Contexts (see Appendix). The various sections of the survey— teacher preparation, 
teaching knowledge and practice, and individual/school variables— correspond to this theoretical 
model. In general, the teacher preparation items function as independent variables; teaching 
knowledge, beliefs, and practices are mediating variables; and personal, classroom, and school 
items serve as control variables. The dependent variables are measures of teacher effectiveness 
that are derived from student learning growth as measured by standardized tests (more about the 
Tennessee methodology later). 

There are three types of questions about teacher preparation on the survey. The first 
type asks about the structural aspects of preservice teacher preparation. Claims have been made 
about the efficacy of a number of structural factors, such as having students complete an 
"extended" (five or fifth year) program (Carnegie Forum, 1986; Holmes Group, 1986; NCTAF, 
1996). Therefore, we include questions about program type and length, asking about alternative 
as well as traditional routes to teacher licensing (Andrew, 1998; Dill, 1996). In a similar manner, 
we developed constructs for other structural factors such as the alignment of teacher preparation 
with the K-12 curriculum (Cohen & Hill, 1998); program coherence (Floden & Buchmann, 1990; 
Howey, 1996); cooperative relations between schools and universities such as PDSs (Goodlad, 
1994; Holmes, 1990; NCTAF, 1996); faculty quality (NCATE, 1995; Holmes Group, 1995); and 
assessments of teacher candidates based on performance criteria (Elliot, 1 996; Wilson, 1 995). 

The second type of question asks about curricular and conceptual aspects of the 
respondent's preparation. Conceptual factors in the preparation of teachers that seem most 
essential are: subject matter preparation (Linn & Baker, 1998; Rowan, Chiang & Miller 1997); 
pedagogical preparation (Darling-Hammond, Wise & Klein, 1995; NCRTE, 1991), and 
preparation for student diversity (Ladson-Billings, 1995; Zeichner, Grant, Gay, Gillette, V alii & 
Villegas, 1998). A similar process was followed in developing induction constructs (Feiman- 
Nemser & Remillard, 1996; Huling- Austin, 1992) and constructs of teacher knowledge, practice, 
and beliefs (Carpenter & Lehrer, 1999; Chazen & Ball, 1995; Fennema & Romberg, 1999; NRC, 
1998; Pressley, 1998). 

The third type of question asks about professional development opportunities once 
respondents started teaching. These teacher preparation questions are followed by inquires 
about teacher knowledge (literacy, mathematics, and pedagogy), teaching practices (use of time, 
materials, and activities), and efficacy beliefs. The final type of questions, which serve as control 
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variables, ask for potentially pertinent demographic and achievement information as well as 
information about the classroom composition and school climate. 

The final part of the theoretical framework is the dependent variable: teacher 
effectiveness. For the Connecticut sample, this variable was constructed by (1) matching teachers 
to students, (2) obtaining measures of student learning gain, and (3) converting those gains into 1- 
5 ratings of teacher effectiveness. Measures of teacher effectiveness were derived from pre- and 
post-tests of the Connecticut Mastery Test (CMT). In Tennessee, the norm-referenced portion 
of the Tennessee Comprehensive Assessment Program (TCAP), given to all students in Grades 
2-8, is the basis for determining teacher effectiveness in five subject areas: mathematics, reading, 
language arts, science, and social studies. TCAP uses the Terra Nova test by CTB 
Macmillan/McGraw Hill, which provides a “comprehensive modular assessment series” of 
student achievement. Measures of student learning attributable to individual teachers are 
determined through the TVAAS. By using scale scores from students' performance on norm- 
referenced tests and a statistical mixed model methodology, TVAAS obtains unbiased estimates 
of the independent influence of teachers, schools, and school systems on student learning. Having 
these scale scores on each student from Grades 3-8 means that students serve as their own 
control. [See McLean, Sanders & Stroup, (1991) and Sanders, Saxton & Horn (1997) for further 
explanation of this methodology]. 

Pilot Study and Sample 

During the 1 999 summer months, items for the survey were informally piloted with focus 
groups of respondents. The purpose of the piloting was to determine the strength of the 
individual items, the clarity of the instructions and items, and the amount of time the various 
sections of the survey would take to complete. Approximately 170 graduate and undergraduate 
education students at two major state universities participated in this pilot phase. 

After revisions were made based on the pilot study and feedback from consultants, the 
survey was administered to beginning teachers in Tennessee and Connecticut. Approximately 
600 teachers responded. The target population was the same in Connecticut and Tennessee: 1 st , 
2 nd , and 3 rd year teachers of Grades 3-8 who taught reading and/or mathematics. The sample of 
165 beginning teachers in Connecticut came from school districts that volunteered to participate 
in the study. All 4 th , 6 th , and 8 th grade 1 st - 3 rd year teachers in those districts were asked to 
answer the survey and to administer pretests and posttests of the state's standardized 
achievement test to their students during the 1999-2000 school year. 

In Tennessee, the sample was derived from statewide mailing lists of beginning teachers 
obtained from the Tennessee State Department of Education, the National Education 
Association, and local affiliates of the Tennessee Education Association, which allowed us to 
over-sample in large, urban school districts. From the Tennessee mailing we received 445 returns, 
a response rate of about 20%. 



Separate databases were constructed from responses in the two states. Data analysis 
includes simple descriptive statistics (measures of central tendency and variation), factor 
analysis, first-order correlations, and multi-variate analysis. Among the questions the survey 
enabled us to address were: 

• Do specific types of teaching knowledge, beliefs, and practices positively impact 
student learning? 

• Do teacher preparation and induction programs improve teaching practice and, 
consequently, student learning? 

Some Preliminary Findings 

The estimates of beginning teacher effectiveness in Tennessee were more variable in the 
area of mathematics instruction than in reading, language arts, science, or social studies. Because 
of that, we focused our analysis on predictors of effectiveness in the teaching of mathematics. 

Based on estimates from the 1999-2000 school year, the two strongest predictors of 
effective mathematics instruction were a sense of teaching efficacy and, what we are currently 
calling, "adaptability" or "flexibility" in the teaching of mathematics. 

The Efficacy Scale is composed of seven items (D7, 1,6, 11,3,5, and 2) from the survey 
instrument. The anchor to that scale is item #7, "Teachers can do very little to overcome the 
effects of poverty on student learning." Items 1 & 7 are our versions of the Rand General 
Efficacy scale. Their original question was "When it comes right down to it, a teacher can't do 
much because most of a student's motivation and performance depends on his/her home 
environment". We thought the question would function better if it separated motivation and 
performance. Our Item #3 is the Rand "Personal Efficacy" item that is commonly used in other 
studies" "If I try hard, I can get through to even the most difficult or unmotivated student." 

The Math Adaptability/Flexibility scale is also composed of seven items (F2c, 2d, 21, 2j 
and F3h, 3i, 3j) about both teacher and student activities during mathematics instruction. The 
anchor items are teacher frequency in "applying mathematics to real-world problems" and 
"modeling different learning strategies for students" and student frequency in "exploring different 
methods to solve a mathematical problem" and "using manipulative materials or models to solve 
problems." These seem to be efforts to help students make sense out of mathematics and stand in 
sharp contrast to the "drill and lecture" scale that was negatively related to student learning. 

Items on that scale were, for example, presenting information, drilling students on computational 
skills, and having students work individually on problems from a textbook or workbook. 

How do beginning teachers learn to be flexible and adaptable in their teaching of 
mathematics and to help students explore different methods of problem solving? Part of the 
answer seems to be their teacher education programs. The primary predictor of the MathFlex 



variable was the Math Preparation Scale (C39-51), a finding that supports the claim that Lee 
Shulman and others make about the importance of pedagogical content knowledge. Beginning 
teachers were more likely to help their students make sense out of mathematics if they answered 
that their teacher preparation program prepared them "well" or "very well" to do things like 
"teach connections among mathematical ideas" and "teach mathematical problem solving skills." 

Mentor Frequency (G5) also had a positive impact on Math Effectiveness. Those who 
reported being assigned an experienced mentor/teacher who gave them frequent (monthly or 
weekly) feedback on their teaching, classroom management, and instructional planning had a 
stronger positive impact on student learning than those who never or seldom received any 
feedback or encouragement. 

In summary, our basic findings indicate that beginning teachers are more successful in 
their teaching of mathematics if they believe teachers can have an impact on student learning, help 
students make sense out of mathematics, were taught how to do this in their teacher preparation 
program, and received continued support in their first years of teaching by an experienced 
mentor. 



Other preliminary findings include: 

• Candidate Assessment (B 1 7-22) is a strong predictor of MathFlexibility / Adaptability, 
but disappears in the partial regression analysis, where C42 ("my teacher education 
program prepared me to teach connections among mathematical ideas") becomes the 
strong predictor. 

• BIO (perception of preparation in mathematics) is negatively related to teacher 
effectiveness. Self-perception of "strong mathematics" background does not seem 
accurate; however, B7 (strong disciplinary preparation) is positively related teacher 
effectiveness in mathematics. 

• H2a (measure of student absence) & H4h ("The school provides extra assistance to 
students having trouble academically") are positively related to gain. Not surprisingly, 
kids who are in school and who get extra help do better than kids who are absent a lot and 
don't receive a lot of help. H4a ("The school has well defined learning expectations for all 
students") also has a positive impact on student learning. 

• Items G2 and G3 : Professional Development Content (strong focus on mathematics 
and/or reading) are negatively related to student learning growth in science. This might 
indicate that a school is placing such strong emphasis on improving teaching and learning 
in these areas that the science curriculum is being ignored. It is conceivable that science 
isn't even being taught very often. 
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THEORETICAL MODEL 



THEORETICAL MODEL 
Beginning Teacher Preparation Survey 



Overall Relationship ofVgrigbles '- 

Teacher Preparation Factors (#2-15) -> Knowledge, Beliefs & Practices (#20-30) 

Literacy Beliefs & Teaching Knowledge (#22-23, 30) — » Literacy Practices (#26-27) 

Mathematics Beliefs & Teaching Knowledge (#24 &25, 30) — > Mathematics Practices (#28 &29) 
Efficacy Beliefs (#20 & 21) — » Teaching Practices (#26-29) 

Teaching Practices (#26-29) — > Teacher effectiveness/Student learning 
Relationship. Amons/ Within Teacher Preparation Variables : 

Structural Factors (#2-7) -> Conceptual Factors (#9-13) 

Structural Factors (#2-7) + Conceptual Factors (#9-13) — > Overall Factors (#14-15) 

Faculty Characteristics (#4) — > Field Experiences (#5a) + Theory/Practice (#5b) + Candidate 
Assessment (#6) — » Coherence (#3) 

Coherence (#3) + Field Experiences (#5a) + Faculty Characteristics (#4) + Theory/Practice (#5b) 
— » Alignment (#7) 

Licensing Route (#1) -4 Program Type (#2) — » Program Quality (#14) — > Program Impact (#15) 
PD Support, Focus & Characteristics (#16-18) -4 PD Impact (#19) 

Relationship Among Belief/Knowledge/Practices Variables : 

Orientations toward subject matter (#23 &24)— ^activities & materials used (#26, 27, 28, 29) 
Materials (#26 & 28) — > Activities (#27, 29) 

Knowledge of pedagogy & subject matter (#22, 25, 30) — » orientations, materials and activities 
(#23, 24, 26-29) 

Efficacy Beliefs (#20 & 21) — » activities, materials (26-29) 

Control Variables . : 

Teacher Qualities (#31) + Placement (#32 &33) — > Teacher Practices (#26-29) — ^Teacher 
Effectiveness 
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